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The scope of this thesis is twofold. The first is to 
provide a nethodology for the performance measurement of 
datahase systems. The second is the application of this 
methcdclcgy to ' a specific datahase system in an attempt to 
verify the applicahility of this methodology and the 
performance and capacity claims of the database system. 

As a methodology/ the thesis describes the strategies 
and locations for the placement of checkpoints/ the kinds of 
performance data to be collected/ the environment rcr the 
conduct cf the performance measurement and the interpreta- 
tion cf the results. One of the most important contribu- 
tions of this methodology is its capability to obtain actual 
measurement overhead making the presentation of truly accu- 
rate results possible. As an application of this method- 
ology/ we attempt tc validate the performance and capacity 
claims cf an experimental multi- backend database system 
known as J1D3S. Surprisingly/ these claims have been 
valic ated. 
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nmlti-tackecd database system known as MDBS. The basic 
design goal of MDBS is to develop an architecture which 
spreads the work of the database management among multiple 
backends. MDBS makes two basic claims in its design. The 
first is that by increasing the number of backends used as a 
part cf the database computer and by keeping the size cf the 
database constant, the response time of the same trans- 
actions is proportionally decreased. Tne second claim is 
that bj increasing the number of backends and also 
increasing the size of the database, the response time 
remains relatively constant. 

To conduct the performance measurement of MDBS, various 
checkpoints and data collections are incorporated into the 
system. Although all checkpoints and data collections are 
selected to provide the greatest amount of useful informa- 
tion and to incur the least amount of overhead, some over- 
head is unavoidable. A quantitative method for measuring 
the overhead incurred is therefore provided. The perform- 
ance results of MDBS are then accurately adjusted using the 
overhead calculation. In this way, a truly accurate meas- 
urement cf the system may be obtained. 
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As a methodology, the thesis describes the strategies 
and locations of the checkpoint placement, the kinds of data 
on performance collected, the ways in which the performance 
measurement were conducted and the interpretation of the 
results. ^Jaybe of greatest importance is the ability to 
calculate actual measurement overhead allowing for the pres- 
entation of truly accurate results. 

In this thesis, we will focus our attention on the 
response time of the work being done by the database system. 
Ke will net focus on the throughput. Whereas the throughput 
is defined as the average number of user reguests executed 
by the system in a second, the response time of a reguest is 
the time between the initial issuance of the reguest by a 
user and the final receipt of the entire response set of 
this reguest by the user [Bef. 1]. Since the majority of 
the reguests processed by a database system are reguests for 
the retrieval of information, another limitation is made to 
the scope of this thesis. We will focus on the perforitance 
measurement of the resp'onse time of retrieval reguests in 
MDBS. Eopefully, these evaluations will verify the claims 
of WEBS and also provide a general methodology fer the 
performance measurement of any database system. 

E. lEE CEGANIZAIION CF THE THESIS 

This thesis is organized into six additional chapters 
beyond this overview. Chapter II describes our performance 
measurement methcdolocy for database systems. It initially 
discusses the need for such a methodology and continues with 
a separate discussion of both the internal and external 
performance measurements. The chapter then culminates with 
a discussion of the combination of the two performance meas- 
urements, thus providing the methodology to calculate and 
adjust fer internal performance measurement overhead. 
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Chapter III preserits ac overview of the target system, 
MDBS, used to apply the performance measurement methcdclogy. 
A general discussion is given on the attritute-based data 
model, the directory tables, the process structure, the 
message types, and tie execution of a retrieve request. 

The application cf the performance measurement methcd- 
clogy tc the target system, MDBS, is presented in Chapter 
IV. The required modifications to the MDBS software needed 
to perform the measurements is discussed, along with a 
discussion of the modifications to the test environment 
required to control the measurement results. A description 
cf the additional software used for both inter-computer and 
inter-process message processing measurements is also 
provided. 

Chapter V presents the construction of the test database 
and the selection of the requests used in the performance 
measurements. In this chapter, the design of the desired 
test database is first discussed. Due to system 

constraints, only a subset of this design is used for 
testing purposes. The chapter concludes with an analysis of 
the requests used in the performance measurement. 

All the thesis work is brought together in Chapter VI 
with the presentation of the performance measurement 
results. Since the goals cf this thesis are to verify the 
performance and capacity claims of MDBS an'd to provide a 
methcdclogy for the performance measurement of a database 
system, only the tests needed to obtain these gcals are 
performed. In the chapter, results are provided for the 
external and internal performance measurements, and the 
results cf the message processing measurements. 

The thesis ends with conclusions in Chapter VII which 
can be made from the results. It provides a summation for 
the entire thesis and offers suggestions in future work 
which needs to be dene both with the methodology and with 
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the ni€a£ur€Eent cf MEES. It is hoped that this thesis will 
provide a scund nethcdology for the performance measurements 
cf database systems and also provide a definitive verifica- 
tion cf the perfcrmarce and capacity claims of MDBS. 
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Ihe reed for a catabase can best be shown as ccrre- 
si-onding tc the need for infcrmation. A database is a 
repository for the storage of information on a computer, any 
item cr combination of items of which can be easily accessed 
in a relatively short timeframe. A businessman may desire 
all the latest pieces of information to make’ a management 
decision. The combat field commander may desire complete, 
up-to-the-minute reports to arrive at a tactical decision. 

But there are performance and capacity problems that 
must be overcome in providing this information. As an ever 
increasing amount of information is stored in a database, 
the response time of the database system increases notice- 
ably. In addition tc the increase in the size of the data- 
base, there is the effect cf increasing the number cf users 
accessing the system and the number of requests tc be 
processed by the system. Thus the user must select between 
the response time desired and tne information desired, a 
choice the user does not want to and should not have to 
make. The database system needs to be easily upgraded to 
accommodate new users and to increase the database size 
without noticeable change in response time. This is the 
need for the response- time inv a riance in a database system. 

Another problem -is in the timeliness of a response. The 
database system should offer a dependable, constant return 
rate fcr the response to a request. When response time 
becomes unreasonably long due to the computer workload, the 
user will be frustrated. A user desires to have every 
request returned in a timely manner. This is the need for 
response -t ime co nsis te nc y in a database system. 

A final problem is to insure that all necessary infcrma- 
tion is available to the user. Incomplete information is of 
little use. For example, a user may require all requests to 
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have a response within a specified timeframe. This require- 
ment often dictates the maximum size of the database and the 
maximum numter of requests. Therefore, an undesireatle 
limitation is placed on the amount of information available 
due to the limitation on database size. Again, the user is 
forced into making a tradeoff between the response time and 
the available information. Never theiess, despite the 
response time, such information should be made available to 
the user on demand. . This is the need for availabi lity of 
information in the database system. 

Therefore, not only is there a need for a database 
system, there is also a need for a database system with the 
qualities of Invariance, Consistency, and Availability 
(ICA) . Eut ICA can be present in varying degrees in a data- 
base system. The degree of ICA can best be demonstrated by 
the performance measurement of the database system. 

There are two basic types of database systems. The 
first is an online software database management system that 
runs on the host computer system. The second is a database 
machine, which offloads the database functions to a dedi- 
cated backend computer. The current trends in database 
systems involve the design, implementation, and use of data- 
base machines £Eef. 1 through 8]. Net only is there an 
apparent improvement in ICA with a corresponding price per 
performance' advantage, but a datanase machine can free up 
resources at the host, provide support for multiple, dissim- 
ilar hosts, and increase the security on the database by the 
physical separation of the database and the host. Eue 
primarily to the trend toward increasing future use of data- 
base machines, this thesis will concentrate on the discus- 
sion and application of the methodology for measuring the 
database machines. 

A database machine is a database system composed of one 
or more processors, dedicated to performing the database 
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nanagement functions. It is indisputable that a database 
machine is the better of the two types of database systems 
with regards to providing an increase in security, allcwing 
for multiple host support, and freeing up the host 
resources. But there still exists the need to demonstrate 
an improvement in the ICA on a database machine over the ICA 
provided by a host- residen t database system. At the same 
time, there exists a need to compare the invariance, consis- 
tency and availability of several different database 
machines and software systems. Again, this can best be 
demonstrated by measureing these systems. 

Ee spcnse-time consistency is more easily achieved in a 
database machine than in a database system running on the 
host. Whereas the host must share its resources with a 
varying workload, the backend can dedicate its resources for 
database management. Availability frees the Eatabase 
Administrator from the necessity to make tradeoffs between 
the size of the database and the response time. The adminis- 
trator can then load the database with all the necessary 
information regardless of the database size. To achieve and 
verify the response time invariance, of a database machine, 
a methodology to measure its effectiveness must be 
developed. 

Thus, the scope of this thesis is to provide a perform- 
ance measurement methodology for database machines and to 
verify this methodology by verifying the design claims of a 
specific database machine, known as MDBS. Again, these 
claims are related to the guality of response time invari- 
ance; that is, to be able to change the size of the database 
and at the same time maintain constant response time or to 
hold constant the size of the database with the ability to 
reduce the response time. Consequently, the measurement of 
the response time of a database system becomes the focal 
point of our studies. If the response time can be properly 
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te integrated with the target system to produce a sirooth 
transition between target system operation and the operation 
of the tcol. In the first part of this section, we address 
the design considerations cf internal performance measure- 
ment methods. Next, we discuss certain software engineering 
criteria which are applicable to the design of good measure- 
ment tods. _ Finally, we explore the application cf the 
internal performance measurement methodology to a particular 
system. 



a. 


Desi 


gn C 


onsider 




Inte 


rnal 


perf o 


kpcints 


int e 


rnal 


to th 


kpcint 


is d 


ef in 


ed as 


the s 


yst em 


's f 


low of 


urement 


rout 


ines 


which 


em over 


head 


is i 


ntroduc 


target 


syst 


em. 


Additi 


ired tc 


proc 


ess 


tie che 


with the ex 


is ti 


ng targ 



measurement relies 



on 

A 



certain portion of the measurement • software must te inte- 
grated with the target system software to handle events such 
as data storage, message passing, and information processing 
that relate to the checkpoint data. Finally, the existing 
target system software may require additional lines of code 
to handle new cases introduced by the measurement system. 

In most external performance measurement, over- 
head is negligible. However, internal measurement routines 
add signifioant overhead to the database system which cannot 
te disregarded. For internal measurement, we must discover 
ways to reduce the overhead generated by the measurement 
software. We must also he able to measure the overhead 
which cannct be eliminated, sc that the measurements can be 
adjusted accordingly. A very important requirement is that 
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the existicg target system must maintain the capability of 
running unimpeded by tie additional measurement software. 

Consideration must be given to the level in the 
target system where checkpoints may be placed. Some possible 
levels are at the very high level, i.e., the system level, 
the high level, i.e., the program level, the medium level, 
i.e., tie subroutine level, and the low level, i.e., the 
subroutine segment level. Whereas external performance 
measurement only places checkpoints at the very high level, 
internal performance measurement places checkpoints below 
that level. Checkpoints must be placed at a level which 
produces data in sufficient detail to provide the user with 
a basic understanding of the system’s performance character- 
istics. Checkpoints should not be placed at a level sc low 
as to overwhelm the user with detailed data or to interfere 
significantly with system performance. 

For internal performance measurement, the user 
should have the capability to access selected data cut of a 
range cf possible choices. The user should not be required 
to receive information about processes which are net of 
current interest. The interface should be easy to use and 
should net distract the user from his primary goal of under- 
standing the database system by requiring the user to 
remember the unique syntax or semantics of the test inter- 
face. The collected measurements should be made accessible 
to automated processing routines for data reduction. 

h. Software Engineering Criteria 

Measurement software should be designed using 
modern software engineering methods. The resulting software 
should be understandable, maintainable, reliable and compat- 
ible with the target system. Certain software engineering 
methods are of partioular interest. These methods are modu- 
larization, user-friendliness, data abstraction, and 
simplici ty . 
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For modularity, the measurement programs should 
he hierarchically structured with well-defined interfaces. 
Ihe measurement modules should he reusable throughout the 
target sjstem. Modularity allcws the system to be easily 
extended to checkpoints not considered in the initial speci- 
fications. The test interface should present an easy-to-use 
method for obtaining test data- It should automatically 
aggregate data while still allowing the user to access raw 
data. Ihe user should not have to remember the specific 
syntax and semantics cf the test interface. Data abstrac- 
tion should be used sc that subsequent program modifications 
do net result in extensive reprogramming. An appropriate 
choice cf primitives ( data structure and operations ) will 
allow for easy change and produce less system overhead. Ihe 
measurement system should be user-friendly. In addition to 
obeying the simplicity principle, the test interface should 
be forgiving, i.e., system should not crash on bad input, 
provide readable error diagnostics, anticipate errors, and 
guard against these errors. 

c. Issues in the Application to Database Systems 

Application of the internal performance measure- 
ment methodology to a particular database system requires 
that the evaluator understand certain aspects of the target 
system. Ike evaluator must understand the programming 
language used to construct the database system, and the 
structure and operation of the database system. The evalu- 
ator must be prepared to overcome obstacles presented by the 
target system in the course of the implementation cf the 
performance measurement. 

A thorough understanding of the programming 
language is necessary to successfully integrate checkpoints 
and data collection programs into the existing software 
structure. One must be familiar witn the data structures. 
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contrcl structures, caming conventions, and parameter- 
passing mechanisms of the language, in order to ijplement 
the measurement programs efficiently and to minimize their 
overhead. Knowledge cf the language syntax reduces program- 
ming errors and speeds implementation of the measurement 
tools . 

For effective internal performance measurement, 
checkpcints must be ccrrectly placed in the database system. 
Incorrectly placed checkpoints increase overhead and degrade 
perfcrmance measurement by providing useless data to the 
user. The evaluator must possess sufficient knowledge cf the 
target system to allcw for the correct placement of check- 
points. This provides the smooth integration of data collec- 
tion programs, data processing programs and data transfer 
programs into the existing database system. 

Chances are that the target system, when 
initially designed, was not designed with internal perform- 
ance measurement in mind. Instead, the target system was 
designed tc process all reguests efficiently. Integration 
of the internal performance measurement routines may affect 
the target system in unexpected ways. Let us consider two 
examples cf such ways. First, in a messa ge- passing system, 
messages generated ty the measurement programs may require 
modifications to the existing database system so that test 
messages will not be confused with the messages of the data- 
base system. Second, the volume of information generated by 
the measurement programs may overload selected sections of 
the target system. Ihe evaluatcr of the performance meas- 
urement routines must be prepared for such contingencies. 
Ey using the knowledge of the programming language along 
with the knowledge cf the database system, the evaluator 
must be prepeired to offer solutions to the database adminis- 
trator cn how to gracefully integrate the performance meas- 
urement mechanisms into the target system with proper 
modification and without overload. 
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2 . A a €th odoloq7 f or External Pe rf grmance Measurement 

Ihe goal of external performance measurement is to 
provide a collection of methods and tools which will enarle 
us to tetter understand the target system by measuring the 
system as a whole. In this way we can measure the total 
work teirg done ty the database system. We focus on meas- 
uring the response time of the system, .the elapsed time 
between the issuance of a reguest and the receipt of the 
response to the reguest. 

Internal performance measurement has been shown to 
be beneficial in the fine-tuning of a system, and in the 
microscopic examination of the work being performed by the 
system. External measurement provides a quantitative meas- 
urement of the system from a macroscopic view. This allows 
for the comparison of database systems. In the first part 
of the section, we discuss the design considerations of the 
external performance measurement methods. Next, we present 
the software engineering criteria for external performance 
measurement. lastly, we show the application of the 
external performance measurement to a system. 

a. Design Considerations 

External performance measurement should have 
negligible overhead, i.e., the response time with external 
performance measurement should be the same as the response 
time without measurement being performed. This is in fact 
the case. The reason that the overhead is negligible is 
that only two timing checkpoints need to be made. These 
timing checkpoints are placed at the beginning of a request 
and the end of the response to the request, thus providing 
the elapsed time of the response for a request. The timing 
checkpoints need the system time at the start and completion 
of the request. The checkpoints are placed at the very high 
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level to insure a coaplete measurement of the total elapsed 
time - 

There are other issues that must be considered 
to insure that the system being evaluated is as ’pure’ as 
possible. First, the system should retain only these cede 
and messages reguiied for the running of the system. 
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not 
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tools designed to aid the measurement, such as those used to 
create a test database. Such tools should remain in soft- 
ware exterior to the actual database system. 

An obvious consideration is to insure that no 
human interaction is involved in the timings. The system 
software, not the reaction time of the user, is being timed. 
Therefore, the timer should start immediately after a user 
releases the request. The timer should stop immediately 
prior to the display on the selected output device. The 

reason fer stopping the timer prior to display is due tc the 

varying delays caused by the output devices. The speed of 
an output device should not be included into the system 
timing results. 

The final issue involving the placement of 

external performance measurement checkpoints is whether to 
embed the timer code in the system or to call a timer 

routine outside the system. A call to a timer routine 
incurs unwanted timing delays, adding to the impurity of a 
system. If the timer code is embedded, it can be made to 
appear that the system code being tested is embedded in the 
timer code, i.e., placing the timer initialization code just 
prior to the point of the request by the user and the timer 
finalization code just subsequent to the display cn the 
output device. Kith these considerations, an optimal place- 
ment of checkpoints can be selected to take exterral 
perf cr marce timings. 
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t. Software ingineering Criteria 
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primarily designed fci system programming. C is a high- 
level language, that is both powerful and portable. 
Although the support software tools such as database lead 
can be inplemented in a language other than the language in 
which tie database system was written, the evaluator needs 
to be familiar with several different languages if several 
different database systems are to be evaluated. 

The degree of software engineering used in the 
database system design will most definitely facilitate any 
external performance measurement to be done. If the data- 
base system was hierarchically designed using modularity, 
knowledge of the internal workings of the system by the 
evaluator will be minimal. Only the upper level in the 
hierarchy need to be studied for the proper placement of the 
checkpoints. External measurement only requires a macro 
knowledge of the system. This is to insure that the check- 
points are indeed properly placed at the very high level. 

C. TEE COMBINATION OF INTEENAL AND EXTERNAL PEEFCBMANCE 

MEASDBEHENTS 

Separately, internal and external performance measure- 
ments provide a wealth of information to the evaluator. 
Internal performance measurement provides the timings and 
data collections of individual processes in the database 
system. External performance measurement provides the 
elapsed time for the complete reguest. Yet, when the two 
methodologies are combined, there is a synergistic effect to 
the amount of information available to the evaluator. 

The combination of internal and external performance 

measurements is natural. There are benefits to he gained 
for one from the other. For example, we can determine the 

overhead incurred when using internal performance measure- 

ment; first, using the external checkpoint, we collect the 
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elapsed time for processing a particular reguest. This time 
is then compared to the elapsed time of the reguest wten 
hoth irternal and external checkpoints are enabled. The 
difference in the elapsed times of these two measurements 
provides an exact measurement of the overhead incurred by 
the irternal performance measurement software for this 
request. 

Cn tie ether hand, we can use the internal perfermance 
measurement timings to interpret the external performance 
measurement timings. In particular, if a reguest takes many 
hundredths cf a secend as a result of external perfermance 
measurement, the evaluator would want to determine the 
precise distribution of the work. Internal performance 
measurement can answer these questions. By combining the 
two measurements, the whole of the measurement results is 
more meaningful and useful than the individual results. 

In the following chapter, the target system, i.e., I1CBS 
is described. Ihis is the system selected to be evaluated 
using the internal and external performance measurement 
met hcdolcgies presented in this chapter. 
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III. THE MUiT I- BACKEND DATABASE SYSTEM (MDBS) 

Id this chapter we discuss the configuration and theory 
cf operation of the multi-tackend database system (MCBS). 
This chapter has been extracted from papers and reports 
•which have been written on MDBS [Ref. 6, 10, 11, 12]. 

MIES uses two or more identical minicomputers and their 
disk systems to provide a centralized database system with 
support for multiple, dissimilar hosts. One minicomputer 
functions as the controller. User access is accomplished 
through a host computer which in turn communicates with the 
controller. Multiple minicomputers and their disks are 
configured in parallel to serve as backends. The original 
design and analysis of MDBS is due to J. Menon [Ref. 1, 2]. 
The implementation and new design efforts are documented in 
[Ref. 3 through 6]. The database is distributed across all 
cf the taokends. The database management functions are 
replicated in each backend. 

As shown in Figure 4.1, the controller and the backends 
are connected by a broadcast bus. When a transaction is 
received from the host computer, the controller broadcasts 
the transaction to all the backends. Each backend has a 
number of dedicated disk drives. Since the data is distrib- 
uted across the backends, a transaction can be executed by 
all backends concurrently. Each backend maintains a gueue of 
transactions and schedules requests for execution inde- 
pendent cf the other backends, in order to maximize its 
access operations and to minimize its idle time. The 
controller does very little work. It is responsible for 
broadcasting, routing, and assisting in the insertion cf new 
data. The backends do most of the database operations. 
Presently, MDBS is fully operational with a VAX 11/780 as 
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Ihe MDBS Structure. 
3 1 



Figure 4. 1 



the ccntrclJer and tvo PDF 1 1/4hs and their disXs as the 
tackerds. 

KIBS is a message-oriented system. In a message-criented 
system, each process corresponds to one system functicn. 
Ihese processes, then, communicate among themselves by 
passing messages. User requests are passed between processes 
as messages. The message paths between processes are fixed 
for the system. The MDBS processes are created at S 7 stem 
start time and exist until the system is stopped. 

MDBS is designed to perform the primary database opera- 
tions, INSEBT, DELETE, UPDATE, and RETRIEVE. Of these four 
database operations, only the retrieval operation will be of 
concern to us in this thesis. The syntax and semantics of 
the retrieve operation is discussed in Chapter V. Users 
access MDBS through the host by issuing either a request or 
a transaction. A t ransa c tion is a set of requests. A 
request is a primary operation along with a qualification. A 
qua lific at i on is used to specify the information of the 
database that is to b'e accessed by the request. More 
complete definitions cf the MDBS terminology can be found in 
the following section. 

In the remainder of this chapter we first discuss the 
directory structure. Next, we provide an overview cf the 
process structure. Then, a presentation of the message types 
is provided. Lastly, we trace the execution sequence of a 
retrieve request. 

A. TEE ATTRIBUTE-BASED DATA MCDEL 

In this section we discuss the attribute— based data 
model. Next we provide some definitions in order to discuss 
MDBS directory data- We conclude this section by describing 
the tables necessary to maintain the MDBS directory 
information . 
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data is modeled with 



In the attri tute-hased data model, 
the constructs: datahase, file, record, attr ibute-value 

pair, directory keyword, directory, record body, ke^'word 
predicate, and query. Informally, a databa se consists of a 
collection cf files. Each file contains groups cf records 
which ate characterized by a unique set of directory 

keywords. A re cor d is composed of two parts. The first 

part is a collection of at t ribu t e— va lue p air s or keywords. 
An attribute— value pair is a member of the Cartesian product 
cf the attribute rame atd tne value domain of the 
attribute. As an example, <P0PU1ATI0N, 25000> is an 
attribute— value pair having 25000 as the value for the popu- 
lation attribute. A record contains at most one attribute- 
value pair for each attribute defined in the database. 

Certain attribute— value pairs of a record (or a file) are 
called tie dir ec tor y k eywo r ds of the record (file) , because 
either the attribute-value pairs or their attribute- value 

ranges are kept in the dir ector y for addressing the record 
(file). Those at tribute— v alue pairs which are net kept in 
the directory for addressing the record (file) are called 
non— directory keywords. The rest of the record is textual 
informaticn, which is referred to as the record body. An 
example cf a record is shown below. 



( 



<fILE, Census>, 

{ 



<CITY, Mcnterey>, <POPULATION, 
Temperate climate } ) 



25000, 



The angle brackets, <,>, enclose an attribute— value pair, 
i-e., keyword. The curly brackets, {,}, include the record 
body. The first attribut e— value pair of all records of a 
file is the same. In particular, the attribute is PILE and 
the value is the file name. A record is enclosed in the 
parenthesis. For example, the above sample record is from 
the Census file. 
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Ih€ database is accessed by indexing on directory 
keywords using k^word predi cate s. A keyword predicate is a 
three-tufle consisting of an attribute# a relational oper- 
ator (=, ## >/ </ >/ <) , and an attribute value# i.e.# 
FOPDIATICN > 20000 is a keyword predicate. More specifi- 
cally# it is a grea ter-than— or-egual-to predicate. 
Combining keyword predicates in disjunctive normal fcrm 
characterizes a gue r y of the database. The guery 

( FIIE = Census and CITY = Monterey ) or 
( FILE = Census and CITY = San Jose ) 

will be satisfied by all records of the Census file with the 
CITY of either Monterey or San Jose. For clarity# we also 
employ parentheses for bracketing predicates in a guery. 

Eecall that in tCBS there are four types of reguests 
which correspond to the four primary database operations. An 
example cf a retrieve reguest would be: 

RETEIEVE ( FILE = Census and POPULATION > 10000 ) (CIIY) 

which retrieves the tames of all those cities in the Census 
file whose population is greater than 10000. Notice that 
the gual if i cat ion ccnponent cf a retrieve reguest consists 
of twc parts# the guery of two predicates ( FILE = Census 
and PCPUIATION > 100CC ) and the target list (CITY). The 
guery specifies which records of the database are to be 
retrieved. The tar get list specifies the attribute— val ue (s) 
to be returned to the user. A user may wish to treat two or 
more reguests as a tr ansactio n. In this situation# MCBS 
executes the reguests of a transaction without permuting 
them# i.e.# if T is a transaction containing the reguests 
<E1><E2># then MDBS executes the reguest R1 before reguest 
E2. Fitally# we define the term traffic-unit to represent 
either a single reguest or a transaction in execution. 
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Figure 4.2 An Attribute Table (AT). 
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cn them. A sample AT is depicted in Figure 4.2. The 
Eescrip t er - to-Descri p to r -ld Tatle maps each descriptor tc a 
unique descriptor id. A sample DDIT is given in Figure 4.3. 
Note that the pointer shown in Figure 4.3 is not actually in 
the ECU tahle but is shown here for clarity to relate back 
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D1 1 


50001 < POPULATION < 100000 


D12 


100001 < POPULATION < 250000 i 


D1 3 


250001 < POPULATION < 500000 


D14 


CITI = Cumberland 


D21 
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D22 


FILE = Employee 


D3 1 


FILE = Census 


D32 

1 



Dig; Descriptor j for attribute i. 



Figure 4.3 A Descriptor-to-Descriptor— Id Table (EDIT). 

to the AT table cf Figure 4.2. The Cluster— Definitio n T able 
maps descriptor— id sets to cluster ids. Each entry consists 
cf the unique cluster id, the set of descriptor ids whose 
descriptors define the cluster, and the addresses cf the 
records in the clusters. A sample CDT is shown in Figure 
4.4. Thus, to access the directory data, we must access the 
AT, EEII, and CDT. 

Cne cf the key concepts used when designing the test 
database (see Chapter V.) is defining the descriptors which 
are specified in the directory attributes. Thus, we provide 
a brief introducticn to the three classif icatiens of 
descriptors. A t y p e-A descriptor is a conjunction cf a 
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D14, D22,D32 


i A3 1 



Figure 4.4 A Cluster— Eefinition Table (QDT) . 

less- than— o r— equal— tc predicate and a qreater-than-or-equal— 
to predicate, such that the same attribute appears in both 
predicates. An example or a type— A descriptor is as 
follows : 

((fCEULATICN > 10000) and (POPULATION < 15000)). 

A descriptor consists cf only an equality predicate. 

An example cf a type-E descriptor is: 

(FILE = Census) . 

Finally, a t ype— C descriptor consists of the name cf an 
attri^te. The type— C attribute defines a set of type— C 
sub— descriptors. Type-C sub-descriptors are equality predi- 
cates defined over all unique attribute values which exist 
in the database. Fcr example, the type— C attribute CITY 
forms the type— C sub-descriptors 

(CITY=Cumberland) , (CITY=Columbus) 

where "Cumberland" and "Columbus" are the only unique data- 
base values for the CITY. 

C. lEF ERCCESS STEDCTORE 

Currently, MDBS does not communicate with a host 
machine. The absence cf this communication requires that the 
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Figure 4.5 The MEBS Process Structure, 
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1. The Processes cf the Ccntr oller 

The controller is composed of three prcce 
Request Preparation, Insert Information Generation, and 
Processing. Request Preparation receives, parses 

formats a request (transaction) before sending the ferm 
request (transaction) to the Directory Management prcce 
each backend. Insert Information Generation is use 
provide additional information to the backends whe 
insert request is received. Since the data is distrib 
the insert only occurs at one of the backends. Thus it 
determine the backend at which the insert will occur, 
with the cluster and descriptor ids for the insert. 
Processing is used to collect all the results of a re 
(transaction) and forward the information back to the 
computer. 

2 . The Proc ess es of Ea ch B ackend 



lach backend is also composed of three prcce 
They are of course different from the controller proce 
They are: Directory Management, Concurrency Contrcl, 
Record Processing. Directory Management performs Descr 
Search, Cluster Search, and Address Generation. Descr 
Search determines the descriptor ids that are needed 
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Address 



request. Cluster Search liras the cluster ids. 

Generaticn determines the secondary storage addresses neces- 
sary to access the clustered records. Concurrency Control 
determines when the request can be executed. Record 
Processing performs the operation specified by the request. 

E. lEI BEES HESSAGE 3YPES 

In this section we describe the MDBS message-passing 
facilities first described in £Eef. 13]. In the MEBS 
message-passing facilities there are 31 message types and 
cne general message fcrmat (shown in Figure 4.6) . This same 



Message Type 
Message Sender 
Message Receiver 
Message Text 



(a numeric code) . 

(a numeric code) . 

(a numeric code) . 

(an alphanumeric field 
terminated by an end 
cf message marker) . 



Figure 4.6 The General Message Format. 

format is used for each of the three message passing facili- 
ties, namely, messages within the controller, messages 
withir a backend, and messages between computers. Messages 
between computers are divided into two classes, messages 
between backends, atd messages between the controller and 
the backends. Figure 4.7 describes each of the MDBS message 
types. Figure 4.8 describes the abbreviations used. 
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MESSAGE-TYPE NCMBER AND NAME 



1 TRAFFIC UNIT 

eecuest results 

NUMBER OF REQUESTS IN 
i TRANSACTION 
AGGREGATE OPERATORS 
5 EECUESTS WITH ERRORS 
PARSEE TRAFFIC CNIT 
NEW DESCRIPTOR ID 
EACKEND NUMBER 

CLUSTER ID J 
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FOR A REQUEST 
EACKEND AGGREGATE 
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RECORE THAT HAS CHANGED 
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OR FETCH CAUSED BY 
AN UPDATE 
15 DESCRIPTOR IDS l5 
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FETCH 

CLE AND NEW VALUES OF 

ATTRIBUTE BEING MODIFIED . 
20 TYPE-C ATTRIBUTES FOR A 20 

TRAFFIC UNIT 
EESC-ID GROUPS FOR A 
TRAFFIC UNIT 
CIUSTER IDS FOE A 
TRAFFIC UNIT 
RELEASE ATTRIBUTE 
RELEASE ALL ATTRIBUTES FOR 
AN INSERT 
25 RELEASE DESCRIPTOR-ID 25 

GROUPS 

ATTRIEUTE LOCKEE 
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GENERATED INSERTS 
GENERATED INSERTS 
GENERATED INSERTS . 
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MDBS Message Types. 
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Figure 4.7 
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Figure 4.8 MDBS Message Abbreviations. 



Ccmmunication bet^ieen computers in MDBS is achieved by 
using a time— di visicn— mult iplexed bus called the parallel 
communication link (FCl) [Bef. 14], MDBS contains a soft- 
ware interface to this bus for each computer consisting of 
two ccnplimentary processes , The first process, get-pcl, 
gets messages from other computers off the PCL. The second 
process, put-pel, puts messages on the bus to be sent to 
other cemputers. The contrcller and each backend have their 
cwn get-pcl and put-pel processes. 

In the remainder of this section, we give short descrip- 
tions of the definitions of MEBS messages. These defini- 
tions are of the form; 



(message-type number) message— type name: explanation of 

message. 



The descriptions will be given by the process that receives 
the message. These descriptions are in following figures: 
Bequest Preparation (Figure 4.9), Post Processing (Figure 
4.10), Directory Management (Figure 4.11), Becord Processing 
(Figure 4.12), Concurrency Control (Figure 4.13), Host 
processed for Test Interface (Figure 4.14), and Insert 
Information Generation (Figure 4.15). 
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(1) Best Traffic Unit : The traffic unit represents a 

single reguest or transaction from a user at the 
host machine. 

(13) Eecerd that has Changed Cluster ; This message is 
a record which has changed cluster^ Request 
Ireparation will prepare it as an insertion and 
send it to the backends. 

(2S) No More Generated Inserts : This message indicates 
that all the records that have changed cluster as 
a result of an update request have been sent to 
Request Preparation. 



(14) Results of a letch or Retrieve Caused by an Update: 
This message carries the information from a fetch 
cr retrieve tack to Request Preparation to complete 
an update with a type-ill or a type— IV modifier. 



Figure 4.9 Bequest Preparation Messages. 



(3) Number of Requests in a Transaction : Request 
Preparation sends to Pest Processing the number 
cf requests in a traffic unit. This enables Post 
Processing tc determine whether the processing cf 
a traffic unit is complete. 

(4) Aggregate Operators ; Request Preparation sends 
the aggregate operators to Post Processing. 

(5) Requests with Errors : Requests with errors will 
be found in Request Preparation by the Parser and 
sent to Post Processing directly. Post Processing 
will send requests with errors back to the host. 

(11) Results cf a Request from a Backend : This message 
contains the results that a specific backend feund 
fer a request. 

(12) Aggregate Operator Results from a Backend : When 
an aggregate eperatien needs to be done on the 
retrieved records, each backend will do as much 
aggregation as possible in the aggregate operation 
function of Record Processing. This message 
carries those results to Post Processing. 



Figure 4.10 Post Processing Messages 



(6) Earsed Traffic Unit : This is the prepared traffic 
unit sent by Beguest Preparation. 

(2S) Nc More Generated Inserts : This message indicates 
that insert reguest for ail the records that have 
changed cluster as a result of an update reguest 
have beer generated and sent to Directory 
Management. 

(7) New Descriptor Id : This message is a response to 
tie Directory Management reguest for a new 
descriptor id. 

(8) Backend Number : This message is used to specify 
which backend is to insert a record. 

(15) Descriptor Ids ; This message contains the results 
of descriptor search by Directory Management. 

(IS) Cld and New Values of Attribute being Modified : 
Eeccrd Processing uses this message to check 
whether a record that has been updated has changed 
cluster. 

(31) An Update Reguest has Finished ; Record Processing 
signals Directory Management that an update reguest 
has finished execution. 



Figure 4.11 Directory Management Messages. 



(16) Reguest and Disk Addresses: Thxs message contains a 
reguest and disk addresses for Record Processing to 
come up with the results for the reguest. 

(17) Changed Cluster Response: Directory Management uses 
this message to tell Record Processing whether an 
updated record has changed cluster. 

(2S) Nc More Generated Inserts : This message indicates 
that all insert reguests generated as a result of 
an update reguest nave been sent to Record 
Processing . 

(18) Fetch : Fetch is a special retrieval of information 
fox Reguest Preparation due to an update reguest 
with type— IV modifier. 



Figure 4.12 Record Processing Messages. 
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(20) Type-C Attritutes for a Traffic Unit : Concurrency 
Ccntrol takes the attributes in this message and 
determines when Descriftor Search for an attribute 
can be perforied. 

(21) Descriptor— id Groups for a Traffic Unit : 
Concurrency Ccntrol takes the descriptor— id groups 
in this message and determines when Cluster search 
for a request can be performed. 

(22) Cluster Ids fcr a Traffic Unit ; Concurrency 
Ccntrol takes the cluster ids in this message and 
determines wten a request can continue with Address 
Generation and the rest of request execution. 

(25) Eelease Attribute : Directory Management uses this 

message to signal Ccncurrency Control that a 
request has performed Descriptor Search on an 
attribute, and the lock on the attribute held by 
the request can be released. 

(2>i) Eelease All the Attributes for an Insert: Directory 
Management uses this message to signal Concurrency 
Control that an insert request has performed 
Descriptor Search on all the attributes, and the 
locks on the attributes held by the request can be 
released . 

(25) Eelease Descriptor-Id Groups : Directory Management 
uses this message to signal Concurrency Control 
that an insert request has performed Cluster Search 
for a request, ana the locks on the descriptor— id 
groups held by the request can be released. 

(51) An Update Request Has Finished : Directory 

Management uses this message to signal Concurrency 
Ccntrol that an update request has finished 
execution, and all the locks held by the request 
can be released. 

(26) Attribute Locked ; Ccncurrency Control signals 
Directory Management that an attribute is locked 
for a request, and Descriptor Search can be 
performed. 

(27) Descriptor-Id. Group s Locked : Concurrency Ccntrol 

signals Directory Management that the Descriptor-id 
croups needed by a request are locked, and cluster 
Search can be performed. 

(2£) Cluster Ids locked : Concurrency Control signals 

Directory Management that the cluster ids needed by 
a request car continue with address Generation and 
the rest of request execution. 

(25) Request Id of a Finished Request : Record 

Erocessing signals Concurrency Control that a 
non- update request has finished execution, and the 
locks on cluster ids held by the request can be 
released . 



Figure 4.13 



Concurrency Control Messages 



(2) Eequest Results : Contains the results for a request 
after neing collected from all the tackends and 
acgregated, if necessary. 



Figure 4.14 Host Messages. 



(S) Cluster Id : lirectcry Management sends a cluster 
id to Insert Informaticr Generation for an insert 
request. IIG will decide where to do the insert. 

(10) Request for New Descriptor Id : When Directory 

Karagement has found a new descriptor, it is sent 
to Insert Information Generation to generate an id. 



Figure 4.15 Insert Information Generation Messages. 



lEE FXECDTION OF A RETRIEVE REQUEST 



In this section, we descrite the sequence Of acticrs for 
a retrieve request as it moves through MDBS. The sequence of 
actions will be described in terms of the messages passed 
between the MDBS processes: Request Preparation (REQP) , 
Insert Information Generation (IIG), Post Processing (PP) , 
Directory Management (DM), Record Processing (RECP) and 
Concurrency Control (CC) . For completeness, we describe the 
actions which require data aggregation. 

First the retrieve request comes to REQP from the host. 
In the present i npleaentati on, it comes from the controller. 
REQP sends two messages to PP: the number of requests in the 
transaction and the aggregate operator of the request. The 
third message sent by REQP is the parsed traffic unit which 
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IV. AN APPIICAl lOH OF THE METHODOLOGIES TO MDBS 



In the previous chapters we discussed the separate 
topics cf aethodolocies for doing internal and external 
perfcritance measurements of database systems and the 
Kulti— backend Database System (MDBS). This chapter presents 
the application cf these methodologies to MDBS. The initial 
discussion concerns modification to the MDBS software. We 
discuss the decisions made during implementation, modifica- 
tion cf the user interface process, the bacKend processes 
and the controller processes, and the issues resolved during 
implementation. The next discussion centers on the modifi- 
cations cf the MDBS test environment, which includes test 
environment changes and software tools. The final discussion 
identifies measurement programs that were implemented 
outside cf the MDBS environment. 



A. 



TEE MODIFICATION CF THE 



MDBS SOFTWARE 



In this section, we begin by presenting the decisions 
made ccncerning the implementation of internal and external 
performance measurements on MDBS. Next, we discuss the modi- 
fications cf the user interface and the individual MDBS 
processes. We conclude this section by relating issues which 
are resolved during the implementation of the performance 
measurement methodology. 



1 • I mp leme n tat icn D eci s ions 



When 

performance 
made as to 
checkpoints , 



designing and specifying internal and external 
measurement methodologies, decisions must be 
the most advantageous positions to place the 
data collections and data aggregations. These 



48 



decisicns are based cr the need to minimize system overhead, 
and tc provide the appropriate level of detail of the test 
data obtained. Primitives and data structures must be devel- 
oped which will allow the measurement programs to remain 
extensible and which are compatible with existing system 
software. A user interface must be developed which is easy 
to use, should not require the user to possess any special 
knowledge of the interface in order to use- it and should 
maintain data in machine readable form which will allow for 
future expatsion of the performance measurement system. 

dhe following implementation decisions are within 
the bounds of two constraints placed upon us by the current 
implementation of MDES along with two constraints we placed 
cn ourselves. The first constraint concerns the virtual 
memory available to the processes resident on the backends. 
The operating system on the PDF— 11/44 allocates a virtual 
memory of 64 Kbytes. Each of the MDBS backend processes must 
fit into a virtual memory of this size. The additional soft- 
ware added as a result of performance measurement has to be 
constructed so that it will fit in a the very limited memory 
space remaining in each backend process. The second 
constraint concerns the initial MDBS design reguir ements 
which called for a broadcast bus between minicomputers. 
Currently a Parallel Communications Link (PCL) is being 
employed as the inter— com puter message— passing mechanism. 
Messages passed over the PCL are se<auen tially transmitted 
from the sender to the receiver. This difference in opera- 
tion between the PCI and the broadcast bus must be taken 
into account in cur attempt to validate the claims of MDBS. 
Additional performance measurement programs must also be 
written to measure message-passing times on the PCL. 

The third constraint, i.e., minimizing overhead, 
significantly influences our performance measurement design. 
This subject will be discussed in the following paragraphs. 
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Ihe filial ccustraint deals with cur desire to run MDBS ur.im— 
feded t'j the new performance measurement software. When we 
are net evaluating the system, we want to te aile tc run 
MDBS with no overhead incurred hy the additional programs 
and checkpoints of tie performance measurement system. This 
is accomplished by tracketing all performance measurement 
software within special preprocessor instructions which 
allow us to include or omit the performance measurement 
software during program compilation. A definition file is 
created containing flags which are used to determine the 
sections of performance measurement code to be compiled. By 
compiling separate versions, we then have the capability of 
running MDBS without performance measurement overhead or 
with the overhead introduced when we select certain portions 
of the performance measurement software for compilation. 

Communication in MDBS is accomplished by passing 
messaces. frocesses which are resident in the same minieem— 
puter communicate bp using inter- process messages, while 
processes resident in different minicomputers communicate by 
using inter-computer messages. Actions taken by the various 
processes in MDBS are initiated by the receipt of a message. 
Actions end when that message has been processed and any 
resultant messages have been sent. As a message is received 
by a process, the action taken by the process is dependent 
cn the message origination and type. The general MDBS 
process procedure hierarchy is shown in Figure 5-1. 

Ihe highest level of this process is the main proce- 
dure. This procedure receives the next message and based 
upon the originator cf the message, calls a sub procedure in 
the procedure hierarchy. The message works its way down this 
tree of sut procedtres based upon the originator of the 
message and the message type. Ultimately, the message 
arrives at a messag e-handl ing procedure (message handler). 
The message handler has the responsibility of processing the 
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Figure 5. 1 The MDBS Procedure Hierarchy. 

message. In doing sc, it may call other procedures Icwer in 
the hierarchy. MDBS's message oriented approach naturally 
lends itself to checkpoint placement at this level. 
Selecticn of measurement at this level provides the user 
with sufficient processing details while not overburdening 
the user with excessive information. A range of six to 
twelve checkpoints maj be installed in each MDBS process at 
this level. The general approach to the installation of 
checkpoints is shown in figure 5.2. In this installation, we 
insert checkpoints hcth before and after every message 
handler. As a result, we obtain the time cf entry intc the 
procedvre and exit from the procedure. The differences 
between these times is the time it takes to process the 
message. 
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processirg speed of the message— handling routines. The deck 
on the PEP- 11/44 measures time in discrete time intervals of 
cnly cne sixtieth of a second. The clock on the Vax-1 1/780 
measures time in discrete time intervals of only one 
hundredth of a second. In any given time interval, the 
system time may be accessed by the performance measurement 
software. This means that access may occur exactly when a 
time interval is recorded by the system clock or anywhere in 
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hetw€en the recordics of a time interval by the system 
clock. Because of this condition, the variance of the time 
measurement would te approximately twice the smallest 
interval. Ihis variance is significant when it is ccmfared 
to the time it takes a message-handling routine to process a 
message. A method must be developed to reduce this variance. 
Ihe scluticn is to send multiple reguests to the message- 
handling routine being timed, to record the time for each 
request and then tc compute the average of the recorded 
times, thereby obtaining a mere accurate measurement cf the 
true processing time. 

In order to keep overhead to a minimum and to keep 
the performance measurement system extensible and simple, 
we decide tc place minimal performance measurement software 
in at MEES process. Kc processing of test data is done in an 
MDBS process. All test data is sent to the test— interface 
routines for aggregation and storage. Since MDBS is a 
message-hased system, measurement control messages and test 
data are transferred as messages utilizing existing MEBS 
communications routines. A differently-oriented system, such 
as procedure— oriented, would require a different approach to 
ireasurement software communication. 

Ihe installation of the checkpoints requires that a 
method he devised to collect the information obtained by the 
checkpoints. The information could be stored locally, trans- 
ferred tc a central storage location in the minicomputer or 
sent to the test interface for storage. In order tc reduce 
the system overhead introduced by message passing we deter- 
mine that the temporary local storage of data would te most 
efficient. As pointed out previously, one of the constraints 
placed upon the implementation of performance measurement is 
the virtual memory space available at the backends. Storage 
of the test data generated from the checkpoints would have 
to be large enough tc contain sufficient timing information 
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and small enough tc reside in the constrained virtual 
memory space available to a backend process. For our timing 
measurements, the upper bound cn the number of requests sent 
to a message handler at one time is fixed at one hundred. In 
ether words, we assume that measuring a given function more 
than 100 times will ret provide a statistically significant 
difference over measuring that same function exactly 100 
times. Given this upper bound, we decide that a static array 
cf 2C0 records would be small enough to fit in the virtual 
memory of a backend process, yet large enough to held a 
sufficient amount of test data. Figure 5.3 shows the general 
approach to the placement of the performance measurement 
routine (limer) which is called by the checkpoint, accesses 
the system clock and manages the static array. 

Another question that must be answered is the manner 
in which the checkpoints are activated. Should we activate 
only cne checkpoint at a time or multiple checkpoints at 
cncer We determine that activating more than one checkpoint 
at a time cculd introduce error into tne measurement. If cne 
routine (A) which is being measured called another routine 
(B) which is also being measured, the time necessary to do 
timing measurements cn (B) wculd increase the total running 
time of (A) . Because of this we only allow the measurement 
cf one routine at a time. 

Ihe desire tc provide a user interface which is easy 
to use and requires no particular knowledge of test inter- 
face implementation leads us to develop a menu— driven 
system. Ihe modularity of the performance measurement design 
lends itself to easy access via menus. The menu-driven 
system is also compatible with the existing test interface 
system. 

Ihe final problem is how to process and stcre the 
raw test data received from the various processes. We 
require that the user have access to both raw data and 
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Figure 5.3 The Procedure Hierarchy 
with Checkpoints and Timer. 
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compiled as the underlying operating system ( on the mini- 
computer where the controller software resides ) creates new 
versions of these files each time the measurement programs 
are invoked. 



2. Ihe Modifications of the User Interface 



Iricr to the implementation of the performance meas- 
urement methodologies, the test interface process consisted 
of those programs necessary to generate a test datahase, 
load a test database and execute requests against the test 
database. Ihe implementation of performance measurement 
software within the existing software structure of the test 
interface is accomplished by expanding the existing hier- 
archy cf control and by integrating performance measurement 
software with existing test interface software. Figure 5.4 
shows the test interface procedural hierarchy with the 
performance measurement modifications. 

Ihe user selects actions to be performed by trav- 
ersing a tree. At each node, a decision is made as to the 
path to follow. By following certain paths, the user has the 
capability to generate a database, load a database or 
execute the test interface. When the user decides to execute 
the test interface, a decision is then made as to what path 
to follow cn the test interface sub-tree. The user may 
choose a new database to work with, create a new list of 
traffic units, modify an existing list of traffic units, 
select traffic units from an existing list for execution, 
select an existing list so that all traffic units cn the 
list may be executed, display the results of external meas- 
urement cr perform a combination of internal and external 
performance measurement. The user may traverse the tree at 
will moving up and dcwn the tranches to accomplish a wide 
variety cf tasks. 
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Figure 5.4 The Test Interface Hierarchy 
with Perfcimance Measurement Software. 
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Ihe MDBS software in the test interface contains a 
procedure, called by the other MDBS procedures, to execute a 
request (transaction). That is, to forward a request (trans- 
action) to Bequest Ireparation for processing. This proce- 
dure is selected fcr the placement of the external and 
internal performance measuring software necessary to time 
and manipulate requests. External measurements are taken 
from this procedure immediately before £he request is sent 
to Bequest freparetticn for processing and after the results 
are returned from Post Processing. Software is added tc this 
procedure to generate requests to the MDBS processes which 
initialize the message— handling routines for internal 
perfcrmance measurement, generate multiple, identical 
requests in order tc reduce the timing variance (as previ- 
ously discussed) and to generate the test data collection 
message. The number of multiple requests to generate is 
provided tc this routine by a variable defined at compile 
time. This procedure receives the information necessary to 
accomplish these other tasks by sharing a first— in— las t— cut 
stack and a pointer to the top of the stack with the 
performance measurement software. The evaluator interacts 
with the performance measurement software to build a stack 
of internal perfcrmance measurement requests. This procedure 
then draws from that stack, xnitializes the messace- 
handling routine selected by the evaluatcr, generates 
multiple copies of the MDBS request selected by the evalu- 
ator, and generates the request necessary to collect the 
test data from the process which contains the message- 
handler being evaluated. Figure 5.5 shows the relationship 
between this procedure and tie performance measurement soft- 
ware and its data structures. 

In addition to external system timing, ether 

perfcrmance measurement functions provided by the new soft- 
ware include routines which allow the user to 1) select 
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Figure 5.5 The Relationship of the Request Execution 
and the Performance Measurement Interface. 

specific MDBS message— handling routines to be timed/ 2) 
select all nessage— handling routines within a process to be 
timed, 2 ) restrict the timing of backend message— har cling 
routines tc a specific backend or backends and 4) perform 
any combination of the aforementioned selections. The new 
performance measurement software also includes routines to 
control the timing software within the MDBS processes, 
collect raw data transferred to the test interface from 
processes within MDBS, process the raw data into summarized 
form and stcre the data for future use. Other routines are 
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intrcduced into existirg test interface software to aid in 
the tt€ssage-passing reguire iients of the performance measure- 
ment system. 

Modi fi cation of Indi vidual P ro cesses 

Ihe PCL processes within MDBS are modified to pass 
perfcrmance measurement messages. Ail of the remaining MEBS 
processes received identical mcdif ica tion. The send/receive 
portion of every process is modified to include the capa— 
hility of processing performance measurement messages. 
Send/receive is used for inter— process message passing. 
Chechpcints are placed in the MDBS processes at the message- 
handling (low) routine level. A timer routine is placed in 
each process which receives control messages from the test 
interface. An initialization message causes the timer 
routine to initialize the data collection array to zero and 
turn on a selected checkpoint. As MDBS— generated messages 
pass through a checkpoint, the timer routine is called. The 
timer routine accesses the system clock and stores the 
message type and time in an array. A completion message from 
the test interface causes the routine to transmit the data 
collected in the array to the test interface and to turn cff 
the checkpoint which is timed. Figure 5.6 shows the modifi- 
cations made to the directory management process as an 
example cf the implementation of the general modifications, 
shown in figure 5.3. 

Iss ues Besolyed Pur ing the Implementatio n 

KEBS is an experimental database machine. As such, 
it is under constant modification and subject to use by many 
system developers. The MDBS software engineering environment 
requires that versions be used to control program modifica- 
tion, but it is impractical to create new versions cf MEBS 
every time a single program is modified. One solution we 
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The Directory Management Hierarchy 
with Checkpoints/Software . 



implemented is 
someone desired 
to the developer 



the maintainance of an in— use file. When 
to modify a program, the program is copied 
•s private work space. The developer makes 





an entry in the in— use file which indicates who is currently 
modifying that particular program. This method allcwec the 
developer tc modify a program, compile and test the modifi- 
cation ir. an environment away from the main MDBS envircnment 
( in order to avoid cc mpromising a functional system ) and 
return the program tc MDBS upcn completion of testing. This 
methcd avoids the possibility of two developers concurrently 
modifying the same program and the ensuing problems. Machine 
time is also at a premium. There is no easy solution for 
this. Much cf the measurement must be conducted during ncn— 
peak hours such as late evening and weekends. This is neces- 
sitated by the reguirement that tne measurement cf MDBS be 
accomplished in a stand— alcne environment. Since the MDBS 
contrcller is i jplemented on a time— sharing system, the 
entire machine has tc be reserved for performance testing so 
that MDBS cculd be run in isolation. 

The performance measurement system places additional 
demands cn MDBS system mess age— passing software. Except for 
one case, the system responds without protest to this unex- 
pected lead. The message— processing routines of the MDBS 
backends are not designed to handle the transfer of 200 
internal performance-measurement messages from a backend 
process to the controller. There is not sufficient space 
available tc store the pointers required to access this many 
messages. The MDBS programs are easily extended tc account 
for this change in message traffic. 

The MDBS ccntroller resides on a VAX-1 1/780 which 
operates under a time— sharing mode. When inter— cempu ter 
messages are passed cn the PCI, the operating system expects 
a confirmation within a certain time interval. While no 
problems occur during the normal operation of MDBS, the 
large message traffic from the backends to the contrcller 
during internal measurement require more time than that 
alloted to the ccntroller during its quantum on the 
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VAX-1 1/7£0. The result is that the controller processes on 
the VAX are suspended while the backend is still transmiting 
over the FCL. When the PCL receives no response it signals 
an error and aborts. Obviously, this is not a problem when 
the MEBS system runs stand alone. However, such abortion 
does provide more than an inconvenience during the iiplemen— 
taticn of the performance measurement system. 

Currently, MEES utilizes two diiierent type of irir.i— 
computers. This translates into two different operating 
systems, two different text editors, two different compilers 
and two different system clocks which record times in 
differing units. Because of this, performance programs in 
the ccntioller processes and the backend processes are not 
identical. Different access mechanisms for system timers 
must be developed and a routine must be developed to convert 
the times received from the bacxends into the equivalent 
time units of the controller. Additional time and effort 
are required to become sufficiently knowledgeable on the two 
systems in order to begin implementation’ of the performance 
measurement methodology. 

E. TEE KODIFICATIOH CF THE MDES TEST ENVIRONMENT 

In conducting performance measurements , one demands that 
all the measurements be consistent as well as reproducible. 
There should be no inconsistent, unexplainable results. 
Further, the results should be reproducible with re— tuns. 
This section discusses the necessary changes in the test 
environment to insure consistency and reproduciblity . Then 
we present the software tools used to make the testing 
easier ard smoother. 
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• Ngc essar v Charges to tte Test Environment 

The methodologies for internal and external perform- 
ance ireasurement s on a database system have one prerequi- 
site. The results itust not he accidental. These results 
need tc he consistent and reproducible. To achieve consis- 
tency and reproducihlity, we must be able to control the 
test environment. Every scientific experiment requires the 
test environment to be controlled, to insure that all 
factors effecting the experiment are known. 

The experimental MDBS, the system to be tested, has 
its controller processes running under a VAX/VHS environ- 
ment. This requires these processes of the controller to be 
run simultaneously with the other non— system processes in a 
timesharing environment. Under tnis environment, the 
results obtained would be erratic and inconsistent. To 
alleviate this, several preliminary steps are taken prior to 
final testing. The tests are run stand-alone with all other 
logins tc the computer disabled. All processes are given 
the highest possible real-time priority. Swapping cut of 
processes in the wait state is disabled to retain the 
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2 . Software 


Tools- for 
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Environment 







An evaluator should understand the system tc be 
tested, determine the various parameters to be altered, 
specify the various data to be collected, and interpret the 
results. Tedious and busy work, such as modifying the input 
set or the system configuration, can be done manually and 
are time-consuming without proper tools. Nevertheless, 
these modifications are necessary, and can be automated by 
using software tools. 
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In [fief. 9], software has been provided tc generate 
a database and a request set, lead the database, and run the 
request set against the database which can all be used in 
the testing of MDBS. This allows for easy creation and 
modification of the selected database and requests. The 
system software needs to be modified during the testing to 
accommodate such things as changes in the number of backerds 
being used by the system and whether or not internal testing 
is tc be performed. Each change requires a recompilation of 
the system software. To facilitate this change and to 
insure only recompilation of necessary files, the Unix 
•make’ command is used. Briefly, execution of this command 
would check a file created by the author. This file would 
indicate all interdependencies of all files of MDBS. If a 
file has been changed, all other files effected by this 
change will automatically be recompiled and relinked upon 
execution of the 'make* command. In this manner, the system 
could be reconfigured with ease and with the assurance that 
all effected files are changed. Using these software tccls 
for the test environment and with proper control of the test 
environment, the tests are made easier to conduct and 
control and are known to be consistent and reproducible. 



C. AEDITICNAL MEASOBEMENT 

In order tc complete 
passing mechanisms must be 
required tc pass both i 
messages. Although the mea 
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in the next Chapter VI. 
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• Int €r- comput€i M ess ag€ Pr oces sing Mea su rement 



New software does not have to he developed to 
measure tie time required to pass messages on the PCL. 
Programs are provided by the manufacturer of the PCL which 
measure the message-passing time. The evaluator is given the 
capability of specifying which node on the PCL is to receive 
the message, the message length and the number of messages 
to send. The software generates and sends the messages, then 
provides the total time tc transmit the messages tc the 
evaluator. The PCL is implemented as a ring bus. Because of 
this style of i nplementation, we decide to send messages 
from cne selected node to itself. The times obtained are an 
upper bound to the i r ter-co mpu ter message passing time. 

2 • Int er-proce ss Messa ge Processing Me asuremen t 
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the tiae necessary tc alter the priority merely gets the 
time, alters the priority a selected number of times, then 
gets the time again. There are two programs necessary to 
determine the time tc process switch. They are identical to 
the two frcgrams mentioned above except that the number of 
messages between process switching is set to one. Utilizing 
the above programs we are able to obtain the inter-process 
message— passing times on both the PDP— 11/44 and the 
■VAX-11/7gO. The next chapter will discuss the selected 
datahase, reguest sets, and procedures taken to run the 
actual benchmark of KIBS. 
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V. IHE BE MCHMAE K OF ^BS 

lie ccnstruction cf the test datax>ase and the selection 
of reguests are very important in the performance measure- 
ment cf a test database system. The test database should be 
representative of a real database, but, as presented in 
£Ref. 7], the test database should be modeled independent of 
any specific database. Both the Test database and the 
requests selected should be properly modeled to allow for a 
complete exercise of the target system. At the same time, 
parameters must not be selected randomly, but rather should 
be created to provide the evaluator flexability and ease of 
evaluation. In this chapter, we first describe the manner 
in which the test database is modeled. We then describe the 
request set which is used in the performance measurement 
experiments . 

A. TEE EEIICTED DATAEASE 

Since MEBS is an experimental database system, it is 
constantly being improved and enhanced. For this reason, 
the test database is designed to facilitate measurements by 
being easily expandable. A. distinct ion will be made in the 
following discussions between the design of the test data- 
base, which allows for future measurements, and the actual 
implementation of the test database used in the measurement 
experiments . 

1 . The Des ign of the M ode l Databas e 

Several factors must be considered in the design of 
a model database. Since the system being measured can be 
configured with either one or two bacJcends, the 'work' 
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TABLI I 

The Benchmark Configuration 



Test 


Nc. of Backends 


Mbyte/backend 


A 


1 


IX 


B 
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cr block, MDBS processes information 
memory using a 4Kbyte logical track. 
4Kbytes, we recommend ccnstructirg the 
sizes of 200 bytes, 400 bytes, 1C00 
s [Bef. 7], This gives a range of 2 to 
k. This also creates an environment 
databases, corresponding to the four 
e generated and tested for each configu— 
e I. Table II gives the correspcrding 
records and blocks. 



TABU II 

The Reccrd'-and— Block Relationship 
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400 1 


• ' 10 


1000 


4 


2000 


2 



As described in Chapter III, the target s 
stores records in clusters. Five cluster categories 
been selected for use in the creation of the model data 
The distinguishing characteristic of a cluster categc 
the number of blocks used to store the records in 
particular category. Table III outlines the sizes of 
of the five cluster categories. One final note, the n 
of blocks per cluster must be even. Thus, when the n 
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c£ tackecds is ircreased from cne to two with the number of 
records in the database remaining constant, we are guaran- 
teed that each backend will have the same number of blocks 
per cluster. for example, when the cluster category is 
small, each backend would have one block for the particular 
cluster, insuring an even distribution of the database. 



TABLE III 

The Cluster Arrangement 



Clusters Category 


Blocks/Cluster 


s mall 


2 


small-medium 


4 


medium 


6 


medium- large 


8 


large 


10 

_ 1 



Combining the data in Tables II and III, we can 
construct a matrix cf data which represents the number of 
rec-ords per cluster category. Table IV, indexed using the 
cluster category and the record size, details this informa- 
tion. The number cf records per cluster is obtained by 
multiplying the Records/Block results from Table II by the 
corresponding Blocks/Cluster results from Table III. 

The remaining considerations when developing a test 
database involve the specification of the directory struc- 
ture for the particular record type. In MDBS, a record 
template, which describes the record structure is defined. 
The record template defines the number of attributes in the 
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TABLE IV 

The Heccrds per Cluster Category 



Number of Records per Cluster 
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record, and for each attribute, the attribute name arc the 
attribute type (either integer or string) . Given a record 
template, the directory and non— directory attributes are 
specified. For each directory attribute, a descriptor type 
and descriptor ranges are defined (see Chapter III) . , 

2- Ihe Implementat ion of the Model Dat abase 



This section examines the implementation decisions 
made when specifying the test database and the testing 
strategy. Ihe current version of MDBS, the primary-memory— 
based directory management, stores the directory tables, 
i.e., the AT, DDIT, and CDT , in primary memory. Given the 
primary memory limitations of the backend, we are forced to 
limit the variables mentioned in the previous section. Cur 
first decision is to limit the size of the test database to 
a maximum cf 1000 records per backend. Table V displays the 
configurations which are used in the performance measure- 
ments of MEES. 
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TABLE V 

The Measurement Configurations 



TEST 


No. cf Backends 


Records/ Backend 


A.E 


1 


1000 


E .E 


2 j 


500 1 


C.E 


2 


1000 


A. I 


1 


1000 1 


B.I 


2 


50 0 



Five different system configurations are needed for 
the MEBS performance measurements. Tests A. E, B.E, ana C.E 
are conducted withcut internal performance software in 
place. Test A. £ configures MDBS with one hackend and one 
thousand records in the test database. Test B. E configures 
MDBS with two backends and one thousand records split evenly 
between the backends. Test C.E also configures MDBS with 

two backends, but, the size of the database is doubled to 
two thousand records. Test A. I and B.I are conducted with 
internal performance software in place. Test A. I configures 
MDBS with one backend and one thousand records in the test 
database. Test B.I configures MDBS with two backends and 
cne thousand records split evenly between the backends. 
Using these five configurations, the verification of the 
MDBS performance and capacity claims is simplified and the 
performance measurement methodology of computing the 
internal measurement overhead is facilitated. 

Cur second decision fixes the recora size at 200 
bytes. The 200 byte record minimizes the primary memory 
required to store the record template. In actuality, a 
record of 198 bytes is used. The record consists cf 33 
attributes, each requiring 6 bytes of storage. The record 
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template file used in our experiments is shown in Figure 
6.1. Cf the 33 attributes listed, INIEl and INIE2 are 
directory attributes. tIULlI and STROO to STR29 are non— 
directcrj attributes. 

In cur next decision, the descriptor types and the 
descriptor ranges for the two directory attributes, INIEI 
and INTE2, are defined in the descriptor files (see Figure 
6.2) . Ihe values for INTE1 are classified by using five 
type-A descriptors, each of which represents a range cf 200. 
Ihe values for INIE2 are also classified using type-A 
descriptors. The first twenty— three ranges for INTE2 cover 
40 values, with the last range covering 80 values. Ihe 
non— uniformity of the INTE2 descriptor ranges is caused by a 
size constraint in tie Concurrency Control process. 



Attribute Name 

INIFl 

INII2 

MOIII 

STFOO 

STEOl 
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m 

STB29 



Attrxbute Type 

integer 

integer 

string 

string 

string 



s tring 



Figure 6. 1 



Ihe Record Template File. 
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Attribute Name 


Descriptor Type 


Descri ptor 


Range 
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680 
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720 
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-> 


760 






761 


-> 


800 
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-> 


840 
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-> 


880 
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-> 


920 






921 


-> 


1000 



Figure- 6.2 The Descriptor File. 
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Figure 6.3 The fiecord File. 
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Bequest Number 


Retrieval Request 


1 


(INTE1 = 10) or (INTE1 = 230) 


2 


(INTE2 < 250) 


3 


(INTS2 < 500) 


4 


(INTE1 < 1000) 


5 


(INTE1<200) or (INTE1>801) 


6 


(INTE1<400) or (INTE1>601) 


7 


(INTE1 < 201) 


8 


(INTE1 < 401) 


9 


(INTE1<201) or (INTE1>800) 


The Target Attribute-Values for Each: 


(INTE1 ,INTE2 ,MULTI ,STR0 0 , S IRO 1 , STRO 2 , S TRO 3, STB 0 4 , 
STEC5, £TR06,STRC7,STE0 8 , SIR 09 , STR 1 0 , SIR 1 1, SIR 12) j 

1 j 



Figure 6.H The Betrieval Eeguests. 

241 tc 280, is retrieved. In the third request, 1/2 cf the 
database is retrieved. Thirteen of the 24 clusters must be 
examined. All records in each of the first twelve clusters 
are returned. Only 1/2 of the thirteenth cluster, defined 
by the range from 481 to 520, is retrieved. The system 
searches only fcr records having values in the range from 
481 tc 500 in this cluster. 

The entire database is examined in the fourth request. 
The fifth request retrieves 2/5 of the database. The query 
is divided into two predicates, to obtain all records from 
the first five clusters, and the last four clusters. The 
sixth request is a retrieval cf 4/5 of the database. Again 
the querj is divided into two predicates, to obtain all 
records from the first 10 clusters, and the last nine 
clusters. 
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Ih€ seventh and eighth 
Ihe seventh reguest e^jamine 
record tc he retrieved from 
records frcm the first fiv 
examines 15 clusters/ r 
retrieved fiom the 11th clu 
the first ten clusters, 
similar tc the fifth regues 
ten additicral clusters mus 
records vith INTEl values o 
the ten additional clusters 
nine clusters, like the fif 
this retrieval. Tahle VI, 
clusters examined versus 
retrieved, is a synopsis 
tabular form. 

The reguest set in Fi 
representative of a ccmpreh 
The goal is not to exhaust 
Rather, we focus on appl 
methcdclcgy to MDBS to val 
capacity claims of the syst 
are sufficient for such a 
these line reguests, i-e. 
number in subseguent discus 



recuests are similar in intent, 
s 10 clusters, reguiring cnly 1 
the 6th cluster and needing all 
e clusters. The eighth request 
eguiring only 1 record tc be 
ster and needing all records from 
The ninth and final request is 
t. But unlike the fifth request, 
t be examined. Only twc of the 
f 201 and 801, are retrieved from 
. All records in the remaining 
th request, are also obtained by 
a presentation of the number of 
the percent of the database 
of the previous discussicr. in 

gure 6.4 is not intended tc be 
ensive and complete request set. 
ively measure and evaluate MDBS, 
ying the performance measurement 
idate the basic performance and 
em. We feel that these requests 
validation. We will refer to 
, retrievals, by their record 
sicn. 
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TABU VI 



The Nuffiher of Clusters Examined 
and the Percent of the Database Retrieved 



Ee5uest 

Number 


Number of 

Clusters 

Examined 


Volume of 
Database 
Retrie ved 


1 


2 : 


2 records 


2 


7 


25% 


3 


13 ' 


50% 


4 


24 (all) 


100% 


5 


9 


40% 


6 


19 


80% 


7 


10 


20% + 1 record 


8 


15 


40% + 1 record 


9 


19 


40% + 2 records 
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VI. THE TEST EESOLTS 



Id this chapter, we present the results obtained from 
the performance meascrement cf MDBS. MDBS is currently 
configured with the primar y-memory— based directory manage— 
ment. In this version of MDBS, the directory tables, i.e., 
the AT, DDIT, and CDT, are stored in the primary memory. We 
expect to achieve different results when version F, the 
secondary— memory-based directory management is implemented. 
The test interface is utilized to send the retrieval 
requests discussed in the previous chapter to MDBS for 
processing. Each reguest is sent a total of ten times per 
database configuration. The response time of each request is 
recorded. After some trial runs, we compute the standard 
deviation. We determine that ten repetitions of each request 
is sufficient to provide the desired accuracy. 

The internal processing times of the message— han dling 
routines which are tsed to process a retrieval request are 
also timed. Retrieval (1) and Retrieval (2) are selected to 
conduct internal timing. These requests are selected since 
they retrieve the smallest portion of the test database and 
the processing time for each request is minimal. Recall that 
each message— handling routine is timed independently cf all 
ethers and that each routine must process multiple requests 
so that an accurate average may be computed for the time 
required tc process that request type. Sixteen message- 
handling routines are required to process a retrieve 
request. If we send twenty requests to each routine, a total 
cf 32C requests must be processed by MDBS. Based on these 
figures, the time required tc conduct the internal perform- 
ance measurement of a retrieval that has a response time cf 
twenty seconds will be approximately 107 minutes. This 
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figure dees not include the administrative time required to 
process the internal neasurement data. For this reason, we 
limited the internal performance measurement reguests to 
Eetrievals (1) and (2). 

Additionally, we also limited the number of repetitions 
per message handler to twenty. This is done to reduce the 
processing time per message handler. However, this decision 
reduces the accuracy of the internal performance measure- 
ment, from ten— thousands to hundredths of a second. Thus, 
the internal performance measurement times provide only a 
rough estimate of the time required to handle the respective 
messa ces . 

The first section of this chapter contains the external 
timing results obtained from our measurements. He also 
discuss the performance and capacity improvements obtained 
by adding backends. In the second section we present the 
results from internal performance measurement. The final 
section examines the inter-process and inter— computer 
message processing times. One final 
measurement presented in the tables 
expressed in seconds. 



note, the units of 
of this chapter are 



A. TEE IITEENAL PEBFCBMANCE BESOLTS 

Table VII provides the results of the external perform- 
ance measurement of EDBS without the internal performance 
measurement software. There are three parts to Table VII. 

Each part contains the mean and the standard deviation of 

the response times for Eetrievals (1) through (9), which are 
outlined in Chapter V. The three parts of Table VII repre- 
sent three different configurations of the MDBS hardware and 
the database record capacity. The first part has MIES 

configured with one backend and the database loaded with 

1000 records. The second part has MDBS configured with two 
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TABLE VII 
Ihe Response Time 

Hithcut Internal Performance Evaluation Software 



Request 

Number 


One Backend 
IK Records 




Two Backends 
IK Records 




Two Backends 
2K Records 


mean 


stdev 




mean , 


stdev 




mean 


stdev 


1 


3. 20 8 


C.C189 




2. 051 


0.0324 




3. 352, 


0 . 0 28 2 


2 ] 


13.691 


0.0255 




7.511 


0.0339 , 




14. 243 


'FToTsi, 


3 


26. 492 


0.0244, 




14, 164 


0,0269 


r 


26.737 


C. 0405 


4 i 


52. 005 


0. C539, 




26. 586, 


0.0294 




52. 173 


’^0*33i 


C 


2 1 . 44 9, 


C.0336 




1 1 ,309 


0.0375 




~2T7550' 


*c70 237. 


6 


4 2. 235 


C. 0326i 




21.622 


0. 0424 




42.287 


cT0400 


- 


1 2.28 5* 


C.0408, 




6. 642 


0.0289; 




12. 347 


07C371 


8 


22. 532 


0.02961 




1 1.764 


0.0300 , 




22. 583 


’o7oTio 


9 ] 


23, 913^ 


C.11 15| 




12, 624 i 


0 .0350 




24. 169* 


’070781 



I 

I 



tackends, with the database containing 1000 records, split 
evenlj between the backends. The third part has HDBS config- 
ured with two backends, with the datanase doubled to 2000 
records, also split evenly between the backends. In latle 
VII we notice one data anomaly, the standard deviation for 
request (9) in the cne-backend-with— 100 0— records configura- 
tion, Since we did not conduct an internal performance 
measurement on this request, we are not sure what causes 
this skewed standard deviation, and hence will not attempt 
to cffer an explanation of this anomaly. 

Given the data presented in Table VTI , we can now 
attempt to verify cr disprove the two MDBS performance 
claims. We begin by calculating the response— time improve- 
ment cf MDBS. The r e spo nse- tim e impro v emen t is defined to be 



82 



I he 

Eespcnse— Time 
Ircfrovement 



= 100 % - 



The Response 
Time of 

Configuration A 



The Response 
Time of 

Configuration B 



* 100 % 



Figure 7. 1 



The Sesponse-Tiae— Improvement Calculation. 



the percentage imprcvemen 
request, when the regues 
opposed to one hack end an 
database remains the same, 
used to calculate the r 
particular request, where C 
ends and Configuraticn A re 
Table VIII we present the 
data given in Table VII. 
improvement is lowest for 
retrieval cf two records of 
the re spcnse— time improv 
retrieves all cf the da 
approaching the upper bound 
find that the respcnse-ti 
number of records retrieved 
a hypothesis that even if t 
time improvement will rema 
40 and 50 percent) level. 

Next we calculate the 
The resFcns e— t ime reduction 
tion in response time of 



t in the response time cf a 
t is executed in n backends as 
d the number of records in the 
Figure 7.1 provides the formula 
espcnse-time improvement for a 
onfiguration 3 represents n back- 
presents one backend. Thus, in 
response— time improvement for the 
Notice that the response-time 
request (1) , which represents a 
the database. On the other hand, 
ement of request (4) , which 
tabase information is highest, 
of fifty percent. In general, we 
me improvement increases as the 
increases. This seems to support 
he database grows, the response— 
in at a relatively high (between 

res ponse— time reduction cf MDBS. 

is defined to be the the reduc- 
a request, when the request is 
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TAEIE VIII 



The RespoDse— Time Improvemejit Between 
1 and 2 Backends (External Measurement Onlj) 



r 

E eguest 1 
N umber 


Response Time 
Improvement 


1 


36.07 


2 


45.14 1 


3 j 


46.53 j 


4 


48.94 


c. 


47.27 


6 


48.81 1 


7 


45.93 


8 1 


47.79 1 


9 


47.21 


IX Records 


No 


Internal— 


teasurement Software 


1 


J 



executed in n backends containing nx number of records as 
opposed to cne backend with x number of records. Figure 7.2 
provides the formula used to calculate the the response-time 
reducticn for a particular retrieval reguest, where ccnfigu— 
ration A represents ere backend with x records and configu- 
ration B represents n backends, each with x records. In 
Table IX we present the r espense— time reductions for the 
data given in Table VII. Notice that the response-time 
reducticn is worst for reguest (1) , which represents a 
retrieval of two records of the database. On the other hand, 
the response— time reductions for the retrievals which access 
larger portions of tte database, reguests (4) and (6), have 
only a snail response-time reduction. In general, we found 
that tie response— time reducticn decreases as the number of 
records retrieved increases, i.e., the response time remains 
virtually constant. Again we seem to have evidence to 
support the hypothesis that, as the size of the database 
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increases, the response— t ime reduction will decrease tc a 
relatively low ( 0.1^ or less ) level. 



The 

Fesf cnse— Ti me 
Eeduction 



= 100 % * 



r , T 
I The Response | 
I Time of J 
i Configuration E | 

L J 



f The Response 1 
1 Time of I 

Conf iguraticr. A J 



Figure 7.2 The Besponse-Time-Reduction Calculation. 



TABLE IX 

The Besponse— Time Reduction 
In Doutling the Database Size 



Reguest 
N umber 


Response Time 
Eeduction 


1 


4.49 


2 


4.03 


3 


0.92 


4 


0.32 


c 


0.47 


6 


0.12 


7 


0.50 


8 


0.23 


9 


1.07 


IK Records on each 


B ackend 


No 


Internal— 


Measure ment Software 


1 


J 



85 



Iatl€ X provides the 
leas urement of MDBS with 
software ic place. There 
part contains the mean an 
response-times for the regu 
outlined in Chapter V. Th 
two different ccnfiguratio 
database record capacity, 
one lacker.d and the databas 
two has MDBS configured wit 
containing 1000 records, s 
Ke did net measure the r 
records distributed over tw 
tional information wculd b 
urements. 



results of external performance 
internal performance measurement 
are two parts to Table X. Each 
d the standard deviation of the 
ests (1) through (6) , which are 
6 two parts of Table X represent 
ns of the MDBS hardware and the 
fart one has MDBS configured with 
e leaded with 1000 records. Part 
h two backends, with the database 
plit evenly between the backends, 
espense times with two thousand 
o backends. We felt that no addi— 
e gained by conducting the meas— 



TABLE X 

The Response Time (in seconds) 

With Internal Performance Measurement Software 



• 

Bequest 

Number 


Cne Backend 
IK Records 


Two Backends 
IK Records 




niean 


stdev 




mean 


stdev 


1 

2 

3 1 

4 
c 

6 i 


3.205 
13. 418 
25.9031 
5C.750 
2C.972I 
41.262 


’o7o436' 

0.0172 

0.0119 

0.0374| 

0.02711 

0.0331 




2.219 
7.401 
13.854 
26.402 
1 1 .244 J 
21.517 


0. 0474 
0. 0277 
0.036 1 
G. 0596 
0. 0528 
0.0575 

j 



An interesting 
response times of 
measurement tests. 



aremaly is discovered when we compare the 
the external and internal performance 
i.e., parts one and two of Tables VII and 
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X for requests (1) through (6) . We actually found a general 
improvement, from 0.15J to 5%, in the response times of the 
requests when the internal performance measurement software 
is part of the MDBS code. One hypothesis is that this is 
due to the manner in which MDBS is implemented on the tack- 
ends. Currently, there is not sufficient physical memory 
availatle on each backend. The result is that disk overlays 
are used to swap in the code necessary to run MDBS. The 
additional internal performance measurement code may cause 
the cperating system to overlay differently, thereby 
benefiting the overall performance of MDBS. We still 
believe that there is an overhead induced by the internal 
measurement code and Table XI provides evidence by demon- 
strating that the response-time improvement achieved by 
adding a backend is net as good as that of Table VIII. 



TABLI XI 

The Response Time Improvement Between 
1 and 2 Backends (With Internal Measurement Also) 



■ 




Request 


Response Txme 


Number 


Improvement . 


. 


30.76 


2 


44.34 


3 


46.52 


4 


47.98 


C 


46.39 J 


6 


47.85 


IK Records 


Ev aluation 


S of twar e 




_ 
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E. lEE INIZENAl PEEECEHANCE EESOLTS 

Table XII provides the results of the irternal perform- 
ance ffeasurement of KIBS for a retrieval request. The tires 
measured for each message-handling routine are given for 
both reguest (1) and (2). The message— handling routines are 
listed with the MDBS process which contains the routine. 
Although the results are given to four decimal places, we 
only trust the accuracy to the second decimal place. The 
reason for this has teen discussed in the introduction to 
this chapter. We are not experts on the MDBS system. He can, 
however, make a few comments on Table XII and we are sure 
that those who are experts can use the results contained in 
Table XII tc draw mere in— depth conclusions on the system. 
He see that the controller processes, i.e., Eeguest 
Preparation and Post Processing, spend very little time in 
processing the retrieval reguest. This is a major design 
goal of MDBS and is necessary to prevent a bottleneck at the 
controller when the number of backends increases substan- 
tially. It appears that this goal is met successfully. We 
also observe that the results obtained from Concurrency 
Control are consistent and cf short duration. This is 
expected since there is only one reguest in the system at a 
time and nc access cententien can occur. These tables should 
then be considered as containing the best— case times. The 
majority of work done in the backend is at Eecord 
Processing. Observing the process timings in Record 

Processing, we see that, for both reguests, the addition cf 
an extra backend reduces the record processing time by 
nearly half. 

C, TEE KESSAGE PEOCESSING BESDLTS 



Table XIII provides some average times relating to 
inter-prccess message passing times on the controller and 
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TABLE XII 

Message Handling Eoutine 
Eiocessing Times for a Retrieval Eeguest 



MIBS 
El ccess 

. 


Message 

Handling 

Eoutine 


Request 
M umter 


One 

Backend 
IK Records 


Twc 

Backends 
IK Records 


Eeguest 


Record Count 


1 1 


0. 0005 


0.C015 ' 


Eref aration 


To Post Proc 


2 


0.0000 


0.0000 




Parse 


1 


0.0200 


oTcTio 




Traffic Unit 


2 




0.0180 


0.C185 




Broadcast 


1 


-- - ^ 

0.0025 


0.0025 




Recuest 


2 


0.0065 


0. 0030 


Ecst 


Ccllect 


1 


0.0465 


0.0250 


Eiocessing 


Results 


2 


0.0890 ] 


0.0813 


Eirectcry 


P aised 


1 


0.0699 


0.0450 


Kanagement 


Traffic Unit, 


2 


0.0925 


0.0491 




Did Sets 


1 


0.0516 


0.0566 




Locked 


2 


0.0566 


0.0566 




Cid Sets 


1 


0.0533 1 


0. 0349 




Locked 


2 


0.0450 1 


0.0433 




Descriptor. 


1 


na 


0.0391 




Ids 


2 1 


na 


0.0558 


Ccncurr ency 


Cids for 


1 


0.0424 


0.0433 


Ccntrcl 


Traffic Unit 


2 


0.0425 , 


0.0433 




Did Sets 


1 


0. 0566 1 


~o7o40s 




Traffic Unit 


2 


0.0508 


0.0516 




Did Sets 


1 


0.0025 1 


0.0016 




Released 


2 


0.0008 


0. 0008 


Eecor d 


Entire 


1 ‘ 


2.6462 1 


1.3775 • 


Erocessing 


Process 


2 


12.7100 


6. 5716 




Eeguest with 


1 


0.0466 , 


0.0433 




Bisk Address 


2 


0.0433 i 


0. 0383 




"1 


— 








Cld 


1 1 


0.0130 1 


0.0148 




Re guest 


2 1 


0.0131 


0.0168 




EIO 


1 


0.0844 1 


0.0865 ' 




Read 


2 


0. 8593 


0.8863 




risk 


1 


0.0799 


o7o7 4~i 




Input/Output 


2 


0.0783 


0.0725 
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TABLE XIII 

Inter— process Message Passing Times 



location 


Time to 
Constr uct 
Messa ge 


Time to 
Receive 
Message 


Time to 
Pass 
Message 


C cnt roller , 
Backend 


0.00249 
0.00830 ; 


0.00267 
0. 004 10 


0. 00520 

0.01250 
1 



the tackend. Messages are transmitted between two processes 
on both the controller and backend. Both the number of 
messages and the message length are varied. On the 
controller, the number of messages is varied from 1 tc 100 
while the message length is varied from 2 to 2000 bytes 
(size of the message buffers in MDBS controller). On the 
backend, the number cf messages is varied from 1 to 50 while 
the message length is varied from 1 to 1000 bytes (size of 
the message buffer in MDBS backends) . It takes the backend 
twice as long to process a message as it does the 
controller. We believe the reason to be hardware processor 
speed. An independent test showed that this relationship, 
cf two to one, holds in how long it takes to process an 
assignment statement cn the respective hardware. 
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lE Table XIV we f;rovide iDformation coccerning the time 
to process inter-computer messages on the PCL. Messages of 
length less than forty are overshadowed by the overhead of 
the ECL, There exists a linear relationship between the 
message length and tte time tc pass a message as the message 
length exceeds 100 bytes. We can therefore expect a linear 
perfcrmance from the ECL for the majority of the MDBS inter- 
computer messages. The next > chapter will contain seme 
concluding remarks and discuss areas for further research. 



TABLE XIV 

In ter-coiputer Message Passing Times 



Message 

Length 

(Bytes) 


Time to , 
Pass 

Message , 


1 

Change 


— 


0.0949 


0.0000 


20 


0.0951 J 


0.0002 


30 


0 .0954 


0. 0003 


4C 


0.0957 j 


0. 0003 


5C 


0. 10 05 


0. 0008 


6C 


0. 10 11 , 


0. 0006 


70 


0.10 18 


0.0007 


8C 


0. 1023 


0. 0005 


9C 


0 . 1029 


0. 0006 


ICO 


0.1056 1 


0.0007 


2C0 


0 . 1136 1 


0.0100 


30C 


0.1238 j 


0. 0102 


4CC 


0 . 1339 


0. 0 10 1 


SCO 


0 . 1439 ] 


0.0100 


1000 


0. 1943 


0. 0504 



VII. THE CONCLUSION 

i. A SUKMAEY OF THE fEBFORMANCE MEASUREMENT METHODOIOGI 

1 t error mar.ce M ea sureme nt Meth odcloq y 

Ihe internal performance measurement methcdclcgy 
provides the strategies and locations for the placement of 
checkpoints. It further provides the kinds of performance 
data to be collected. This information enables a tetter 
understanding of the target system by measuring certain 
capaiilities, such as the time spent in individual 
processes. Using this information of how the system 
performs internally may lead to design modifications or to 
fine-tuning of the system for increased performance. 

2 . Ihe Exte rn al P erfo r man ce M easure ment Met hodology 

Ihe external performance measurement methcdclcgy 
provides the strategies for a macro view of the database 
system performance by measuring the system as a whole. Ne 
focus on the measurement of the response time of the target 
system after the issuance of a request. A test database and 
a test request set is generated using software tools. 

3 . Combining the Inte rn al and Ext ern al Measurement 

M €t hodolcqies 

Ihe natural combination of the internal and external 
performance measurement methodologies is synergistic ir. the 
amount of information that is provided. The overhead 
incurred when using internal performance measurement code is 
accurately determined using this methodology combination. 
Ihe external performance measurement timings can be properly 
interpreted using the internal performance measurement 
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results. Ey combining the two measurements, the whole of 
the measurement results is more meaningful and useful than 
the ircividual results. 



B. A SUHMAEI OF THE CETHODOLOGY APPLICATION 

Ihrillirg and unexpected results are collected when this 
methcdclcgy is applied to a target system, i.e’., MDBS. 
First, the methodology proves itself to be successful in 
attempting to verify the performance and capacity claims of 
MDBS. This results from being able tc collect sufficient 

to make definitive statements 
The application of this methcd— 
clogy to MDES is alsc surprisingly easy. 

A second result, is that the performance and capacity 
claims of MEBS have been validated. These claims are: 1) 

that by increasing the number of backends used as a part of 
the database system ard by keeping the size of the database 
the response time of the same transactions is 

and 2) that by increasing the 
number cf backends and also increasing the size of the data- 
base, the response time remains relatively constant. These 
claims are validated by the results given in Chapter VI. 

These spectacular results provide a wealth of informa- 
tion from which several conclusions can be made. Ke find 
that under MDBS, the respo nse— time improvement increases as 
the number of records retrieved increases. Alsc, the 
response- time reduction decreases as the number of records 
retrieved increases. Though the performance measurement 
results indicate an improvement in the response time cf the 
requests when the internal performance measurement software 
is part cf MDBS code, it is felt that this phenomenon is the 
result of differing system overlays and that the induced 



data on a target system 
concerning its performance. 



constant, 
proportionally decreased. 



overhead of internal 
calculated. 



measureitent code still needs tc be 
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tion, as expected, 
cf wcrk is being dene in 



Ihe results of the internal perforaance measure 
indicate that the controller processes, i.e.. He 
Preparation and Post Processing, spend very little ti 
process the retrieval request. The results obtained 
Concurrency Control are both consistent and of short 

Ihe results also show that the maj 
Record Processing and that 
addition of a backend reduces the record processing ti 
nearly half. We discovered that it takes the backend 
as long tc process a message as it does the centre 
possibly due to hardware processor speed. Finally, 
exists a linear relationship between the message lengt 
the time te pass a message as the message length exceed 
bytes . 



C. EiCCHMINDATIOHS fCE FOTUBE EFFOfiTS 



Future improvements can 
urement methodology by t 
external software tods. S 
a test which will execute a 
pre— determined number of ti 
the results in a fil 
Additionally, since the 
general in use, the meth 
different database systems 
ease cf use, and usefulnes 
ment cf the target system. 

In terms of the applica 
a ccmplete and thorcugh t 
conducted. An exhaustiv 
conducting test with data 
sizes. Further, testing th 
directory attributes, descr 



be made in the performance 
he automation of the exi 
pecifically, the ability to 
pre— determined set of reque 
mes for each request, and cc 
e is a desireable fea 
methodology is intended t 
odology needs to be applie 
tc discover its applicabi 
s in overall performance mea 
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est of the system needs t 
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